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IN THE UNITED STATES ELECTED OFFICE 
OF THE UNITED STATES PATENT AND TRADEMARK OFFICE 
UNDER THE PATENT COOPERATION TREATY-CHAPTER II 

PRELIMINARY AMENDMENT 

APPLICANT(S): Gerhard Niedermair et al DOCKET NO: P99,0100 

SERIAL NO: GROUP ART UNIT: 

EXAMINER: 

INTERNATIONAL APPLICATION NO: PCT/DE97/01 920 

INTERNATIONAL FILING DATE: 02 September 1 997 

INVENTION: SPEECH PROCESSING SYSTEM AND METHOD 

Assistant Commissioner for Patents, 
Washington, D.C. 20231 

Sir: 

Please amend the above-identified application as follows before 
calculation of the U.S. national fee under 35 U.S.C. 371 (c) (1). 
IN THE SPECIFICATION: 

On page 1, please delete lines 1-4 and insert the following: 
-SPECIFICATION 
TITLE 

SPEECH PROCESSING SYSTEM AND METHOD 

BACKGROUND OF THE INVENTION -. 

On page 2, line 8, please change "In order to [. . .]" to -There is a 
greater need for—. 
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On page 2, lines 13-14, please change up to now" to --. In the 
prior art--. 

On page 3, line 1 , please change "[3]" to -the reference by Caspari, 
R., 1993, "Bottom Up Generation from Flat Semantic Structures," Proc. 
5 Deklarative und prozedurale Aspekte der Sprachverarbeitung, Deutsche 
Gesellschaft fur Sprachverarbeitung, Sektion Computerlinguistik, 4 th 
Technical Conference, pp. 1-6--. 

On page 3, line 9, please change "[6]" to --Great Britain reference 
GB 2 119 208--. 

10 On page 3, after line 13, as a separate line before line 14, please 

insert the following heading: 

-SUMMARY OF THE INVENTION -. 

On page 3, please delete lines 19-21. 

On page 4, line 23, please change "development support in the 
15 creation of -developing a-. 

On page 5, please delete lines 9-10. 

On page 5, please delete lines 23-24 and insert the following 
heading and paragraph: 

- BRIEF DESCRIPTION OF THE DRAWINGS 
2 o The features of the present invention which are believed to be novel, 

are set forth with particularity in the appended claims. The invention, 
together with further objects and advantages, may best be understood by 
reference to the following description taken in conjunction with the 
accompanying drawings, in the several Figures of which like reference 
2 5 numerals identify like elements, and in which:- 

On page 6, after line 2, as a separate line before line 3, please 
insert — DESCRIPTION OF THE PREFERRED EMBODIMENTS —. 
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On page 6, line 27, please change "documents [1] and [2]" to - the 
reference by G. Ruske, "Automatische Spracherkennung, Oldenbourg- 
Verlag", 1 st ed., ISBN 3-486-20877-2, Munich, pp. 106-171, 1988 and the 
reference by A. Zell, Simulation neuronal Netze, Edison Wesley, 1 st ed., 
5 ISBN 3-89319-554-8, pp. 137 - 143, 1994--. 

On page 8, line 26, please change "document [1]" to --the reference 
by G. Ruske, "Automatische Spracherkennung, Oldenbourg-Verlag", 1 st ed., 
ISBN 3-486-20877-2, Munich, pp. 106-171, 1988--. 

On page 1 1 , line 30, please change "document [4]" to -the reference 
10 by Naumann, Sven and Hagen Langer, 1994: Parsing, Stuttgart, Teubner, 
1 st ed., ISBN 3-519-02139-0, pp. 155- 166--. 

On page 12, line 14, please change "[. . ■ ]" to -for--. 

On page 13, line 31, please change "document 5" to -German 
reference DE 44 12 309 A1-. 
15 On page 19, line 26, please change "cf." to -see--. 

On page 22, line 2, please change "document [3]" to -the reference 
by Caspari, R., 1993, "Bottom Up Generation from Flat Semantic Structures", 
Proc. Deklarative und prozedurale Aspekte der Sprachverarbeitung, 
Deutsche Gesellschaft fur Sprachverarbeitung, Sektion Computerlinguistik, 
20 4 th Technical Conference, pp. 1 - 6--. 

On page 23, line 4, please change "document [4]" to -the reference 
by Naumann, Sven and Hagen Langer, 1994: Parsing, Stuttgart, Teubner, 
1 st ed., ISBN 3-519-02139-0, pp. 155 - 166-. 

On page 24, line 2, please change "is [sic]" to -are-. 
2 5 On page 25, line 1 6, please change "document [3]" to -the reference 

by Caspari, R., 1993, Bottom Up Generation from Flat Semantic Structures, 
Proc. Deklarative und prozedurale Aspekte der Sprachverarbeitung, 
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Deutsche Gesellschaftfur Sprachverarbeitung, Sektion Computerlinguistik, 
4 th Technical Conference, pp. 1 - 6-. 

On page 37, line 31 , after "module" please insert --VV'--. 

On page 38, after line 7, please insert the following paragraph: 
5 -The invention is not limited to the particular details of the method 

and apparatus depicted and other modifications and applications are 
contemplated. Certain other changes may be made in the above described 
method and apparatus without departing from the true spirit and scope of the 
invention herein involved. It is intended, therefore, that the subject matter in 
10 the above depiction shall be interpreted as illustrative and not in a limiting 
sense.-- 

IN THE CLAIMS: 

On replacement page 40, line 1 , please change "Patent claims" to 
WHAT IS CLAIMED IS: 
15 Please amend claims 1-10 as follows: 

1 . (Amended) [Speech] A speech processing system , comprising: 

at least one of a pl urality of [-having several] speech recognition 
modules and a plurality of [(Ei ? 1=1 ..n) and/or several] speech output modules 
[(Aj, j=1 ..m) that are] , each of the speech recognition modules and speech 
2 o output modules being respectively constructed specifically for a particular 
type of speech recognition and/or speech output[ J; and 

[- having a means (MA)] a selector for selecting at least one speech 
recognition module [(Ei)] and/or speech output module [(Aj)] for a speech 
recognition and/or speech output to be carried out later, dependent on an 
2 5 input signal [(ES) with which it is described] that describes what type of 
speech recognition and/or speech output is further required. 
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2. (Amended) [Speech] The speech processing system according to 
claim 1 , wherein [in which] the [means (MA)] selector for selecting the 
speech recognition module [(Ei)] and/or the speech output module [(Aj)] is 
constructed [in] such [a way] that the speech recognition module [(Ei)] and/or 

5 the speech output module [(Aj)] is controlled by the selector [means (MA)]. 

3. (Amended) [Speech] The speech processing system according to 
claim 1 [or 2, in which] wherein parts of the speech recognition module [(Ei)] 
and/or of the speech output module [(Aj)] that are used in common are 
realized in at least one pre-processing module [(VV)] and/or in at least one 

io post-processing module , respectively . 

4. (Amended) [Speech] The speech processing system according to 
claim 3, wherein [in which] several speech recognition modules of the 
plurality of speech recognition modules [(Ei)] and/or speech output modules 
of the plurality of speech output modules [(Aj)] use common resources. 

is 5. (Amended) [Speech] The speech processing system according to 
claim 1 , wherein the system further comprises [one of claims 1 to 4, in which] 
a dialog sequence control [(DA) is provided] with which a dialog of the 
speech processing system with a user [(B)] of the speech processing system 
is realized. 

20 6. (Amended) [Speech] The speech processing system according to 
claim 5. wherein [one of claims 1 to 5, in which] the input signal [(ES)] is 
produced by the dialog sequence control [(DA)]. 
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7. (Amended) [Speech] The speech processing system according to 

claim 1 . wherein [one of claims 1 to 6, - in which] the speech recognition 

module [(Ei)] is constructed at least for [the] execution of one of the following 

types of speech recognition: 
5 [--] individual numeral recognition, 

[--] Recognition of chains of numerals, 

[— ] Recognition of words from a limited vocabulary, 

[--] Recognition of an individual word with an unlimited vocabulary, 

[--] Recognition of speech spoken in a flowing manner with an unlimited 
10 vocabulary, 

[--] Recognition of predetermined word combinations, 

[--] Keyword recognition, 

[--] Alphabet recognition, 

[--] Sound sequence recognition, 
is [--] Speech recognition, 

[--] DTMF recognition, 

and/or 

[- in which the] wherein a respective speech output module of the plurality of 
speech output modules [(Aj)] is constructed at least for the execution of one 
20 of the following types of speech output: 

[— Output] output of predetermined stored speech stores, 

[-- Output] output of combined individual predetermined stored speech 

stores, 

[— Output] output of words synthesized from stored phonemes, and 
2 5 [« Output] output of DTMF tones. 



8. (Amended) [Method] A method for speech processing, comprising 
the steps of: 

analyzing [- in which] an input signal [(ES) is analyzed (401), with 
which signal it is described] that describes what type of speech recognition 
and/or speech output is further required!,]; 

activating [- in which], dependent on the input signal [(ES)], from a 
set of several speech recognition modules [(Ei r I = 1..n)] and/or speech 
output modules [(Aj, j = 1..m)], specifically constructed respectively for a 
particular type of speech recognition and/or speech output, respectively, at 
least one speech recognition module [(Ei)] and/or at least one speech output 
module , respectively, [(Aj) is activated (402),] and 

executing [- in which] the respective type of speech recognition 
and/or speech output [is executed] with the [selected] activated speech 
recognition module and/or speech output module [(403)]. 

9. (Amended) [Method] The method according to claim 8, wherein 
[in which] a pre-processing [(VV)] of the speech signal to be recognized 
takes place before the speech recognition. 

10. (Amended) [Method] The method according to claim 8 [or 9], 
wherein [in which] after the speech recognition a post-processing of [the] a 
recognized speech signal takes place. 

IN THE ABSTRACT 

On page 43, please delete lines 1-4, and insert the following 
heading: - ABSTRACT OF THE DISCLOSURE -. 

On page 43, line 5, please change "comprises" to —has-. 
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On page 43, line 11, please change "means" to -device-. 
On page 43, line 16, please delete "Sig. Fig. 1". 

REMARKS 

The claims have been amended to place them in proper U.S. form. 
5 No new matter is added by the foregoing amendments. 

Applicant respectfully requests entry of the above preliminary 
amendments prior to calculation of the filing fees. 



Respectfully submitted, 




TReg.No. 27.888) 



io John R/Garrett 

Hill &/Simpson 
A Professional Corporation 
85th Floor Sears Tower 
Chicago, Illinois 60606 

15 (312) 876-0200; Ext. 3078 

Attorneys for Applicant 



Specification 



Speech processing system and method 

5 In standard speech processing systems, e.g. for 

telephone-based applications, the widest variety of 
possibilities are available for verbal input and output. This 
ranges from the recognition of individually spoken numerals 
and a mostly limited number of command words to the 

10 possibility of the input of expressions spoken in a flowing 
manner. The output possibilities range from the output of 
completely recorded expressions to the purely synthetically 
generated acoustic output of texts from e.g. stored phonemes. 
Applications of speech processing systems in the 

15 industrial, public and private sectors are confronted with 

very different demands both on speech input and also on speech 
output. This is due to the large number of application 
possibilities and the verbal tasks thereby to be solved by the 
speech processing systems. 

20 This also holds for speech dialog systems in which the 

speech processing system stands in an arbitrarily constructed 
dialog with the user of the speech processing system. 

Given an application in the area of speech processing 
systems, it can for example be required that at a particular 

25 time a numeral is interrogated individually by the user, while 
at another time within the same dialog a date can be inputted 
by the user, to be processed by the speech processing system. 

Likewise, during output at one point the output of a 
stored speech segment may be sufficient, whereas at a 

30 different point of an application scenario a speech synthesis 
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may be more suitable due to the high variability of the text 

to be outputted. 

In addition, at different times different performance 

features of speech recognizers may be regarded as necessary by 
5 the operator; for example, the interrupt ibility of the system 

output may be desirable, whereas at a different point it may 

be desired that the user listen to the complete output. 

In order to [...] the most efficient possible application 

of the specialized known algorithms for speech recognition and 
10 speech output, and also speech dialog systems with more 

complex recognition characteristics and output characteristics 

than for example for simple numeral recognition or only for 

the isolated recognition of a small number of command words, 

up to now it was standard to create the systems in such a way 
15 that the specific requirements on speech recognition or, 

respectively, on speech output were taken into account by 

means of the particular selection of technical methods and 

specific software measures . 

However, this requires a large expense in expert 
20 personnel, which as a rule must be furnished anew for each new 

situation of application. By means of this procedure, an 

efficient economical use of speech processing systems is 

hindered considerably. 

In the area of telephony, there exist dialog systems in 
25 which a selection takes place e.g. via what are called DTMF 

tones and simple naming of numerals or of some less 

predetermined command words . 

However, the expense for a new development of a speech 

processing system or, respectively, of a speech dialog system 
3 0 with the previous methods and systems is very expensive. 
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From [3] , a method is known for improving speech 
recognition while taking into account various digital coding 
conditions. For a multiplicity of speech recognition modules 
that all use the same speech recognition algorithm, but 
5 adapted as optimally as possible to different channel coding 
methods for received signals, the channel coding method 
respectively used is recognized and the respectively 
corresponding speech recognition module is selected. 

From [6] , a method and arrangement are known for 

10 generating a multiplicity of electrical signals. In this 
arrangement, a multiplicity of speech output units with 
identical construction are combined with one another to form 
the overall arrangement. 

The invention is thus based on the problem of indicating 

15 a speech processing system and a method for speech processing 
with which an optimal context-sensitive use of speech 
recognition modules and/or speech output modules is achieved, 
with a reduction of the required computing capacity. 

The problem is solved by means of the speech processing 

20 system according to patent claim 1, as well by means of the 
method according to patent claim 8 . 

The speech processing system comprises several speech 
recognition modules and/or speech output modules that are 
respectively provided for a predeterminable type of speech 

25 recognition or, respectively, speech output. In addition, the 
speech processing system comprises a means for selecting a 
speech recognition module and/or speech output module. With 
this means, dependent on an input signal supplied to the means 
the respective speech recognition module or, respectively, 

30 speech output module is selected, activated, and the 

corresponding speech signal or, respectively, the required 
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parameters are supplied to the respective module. The speech 
recognition or, respectively, the speech output is then 
executed by the respective module. 

In the method, an input signal is analyzed with which it 
5 is described what type of speech recognition or, respectively, 
speech output is required. Dependent on the input signal, at 
least one speech recognition module and/or at least one speech 
output module is activated for the speech recognition or, 
respectively, speech output respectively described by the 
10 input signal. With the respectively selected or, 
q respectively, activated module, the respective type of speech 

Ifi recognition or, respectively, speech output is carried out. 

*5 Both the speech processing system and the method comprise 

flj a number of considerable advantages in relation to the prior 

m!5 art. 

JL : The requirement for computing capacity or, respectively, 

W processing resources for speech recognition or, respectively, 

ril for speech output is reduced considerably, since, dependent on 

^ the respectively required situation of application, the module 

20 that is optimally adapted for the situation is used for speech 
recognition or, respectively, for speech output. 

The speech processing system can also be used to great 
advantage for development support in the creation of speech 
dialog system [sic] . This speech processing system offers 
25 particular advantages in this area because a rapid and simple 
adaptation of the existing technical speech recognition 
methods and speech output methods to the respective specific 
requirements of the situation of application is possible. 

In addition, a flexible use of the various capacities and 
30 performance features thereof, adapted to the specific dialog 
situation, is achieved. A flexible adaptation of the speech 
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recognition performance and speech output performance to the 
resources available for the users in the specific dialog 
system, e.g. to the available computer capacity, is also 
possible . 

5 The speech processing system thus represents an important 

means for the configuration and adaptation according to need 
of speech input and speech output in the widest variety of 
situations of application. 

Advantageous developments of the invention result from 
10 the dependent claims. 

In a development of the speech processing system, it is 
very advantageous to use particular algorithms, i.e. 
particular parts of the modules, in common, whereby a 
redundant realization of algorithms only actually required 
15 once is avoided. 

It is also an advantageous development that several 
speech recognition modules use common resources, e.g. common 
lexicons for speech recognition. This leads to a considerable 
savings of resources, e.g. memory requirement. 
20 In addition, it is advantageous to provide a dialog run 

control with which a dialog of the speech processing system 
with a user can be realized. 

An embodiment of the invention is shown in the Figures 
and is explained in more detail in the following. 
2 5 Fig. 1 shows the speech processing system with some 

developments ; 

Fig. 2 shows a block diagram in which the dialog sequence 

control system is shown; 
Fig. 3 shows a flow diagram in which a possible method 
30 executed for dialog sequence control is shown; 
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Fig. 4 shows a flow diagram in which the inventive method 
is represented. 
Fig. 1 shows the speech processing system. The speech 
processing system comprises at least one speech recognition 
5 module El, E2 , Ei , . .., En. An index i, an arbitrary number 
between 1 and n, unambiguously identifies the speech 
recognition module Ei. The letter n designates a number of 
speech recognition modules in the speech processing system. 

In addition, the speech recognition system additionally 

10 or alternatively comprises at least one speech output module 
Al, A2 , A j , Am. A second index j, an arbitrary number 

between 1 and m, unambiguously identifies each speech output 
module Aj . A number of speech output modules in the speech 
processing system is designated m. 

15 In addition, the speech processing system comprises a 

means MA for selecting the speech recognition modules Ei or, 
respectively, the speech output modules Aj . The means MA for 
selecting the modules is coupled with the individual modules 
Ei, A j , so that signals can be sent by the means MA to the 

20 individual modules Ei, Aj for the selection of the modules Ei, 
A j , e.g. for the activation of the individual modules Ei, A j , 
or also for transferring parameters that are required by the 
individual modules for the execution of the speech recognition 
or of the speech output . 

25 Speech recognition modules Ei 

Fundamental concepts of speech recognition are described 
for example in documents [1] and [2] . 

The individual speech recognition modules El, E2, En 
are modules that are respectively specialized for a particular 
3 0 type of speech recognition. 
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The individual speech recognition modules Ei can for 
example be realized by hardware or also by individual software 
procedures . 

In the fol lowing , an overview is given (in no way to be 
5 regarded as final) of provided speech recognition modules Ei 
that can be used within the scope of the speech processing 
system. 

- Individual numeral recognizer 

If a speech recognition module Ei is fashioned as an 

10 individual numeral recognizer, it is constructed so as to 

process as input a single speech signal spoken in by the user 
B of the speech processing system, which signal represents a 
single numeral from a selection of numerals. The individual 
numeral recognizers thereby standardly differ from one another 

15 in the modeling of the acoustic pattern. Systems are known in 
which the acoustic pattern of different speakers for an entire 
word must be stored, and, using comparison methods such as 
what is called dynamic programming or dynamic time warping, or 
also using neural networks, the best -fitting word is 

20 determined in the recognition process. Other known modules 

for individual numeral recognition are based on the principle 
of hidden Markov models. In this principle of individual 
numeral recognition, the sequence of sounds of the words 
spoken by several speakers are stored as a state sequence with 

25 variable and predetermined length. In the recognition of the 
words, in this case of the numeral, the smallest difference 
from the stored word sequence is determined. In variants of 
this principle, the sequences of the individual phonetic units 
are stored in hidden Markov models, said units being obtained 

30 from the data of several speakers, as a rule numerals. In 
order to reduce the required computing capacity, in an 
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individual numeral recognizer the fact can be exploited that a 
further sequence cannot follow any of the stored phonetic 
sequences. For this reason, in this case a computing- 
intensive detection of word boundaries is not required. The 
5 vocabulary to be examined is also relatively small, since only 
the numerals need be examined. 
- Numeral chain recognizer 

If a speech recognition module Ei is constructed as a 
numeral chain recognizer, as a rule the speech recognition 

10 module is to be equated with recognizers for continuous 
speech. A number of predeterminable words defined in a 
digital lexicon (numerals in this specific case) can be 
expressed in an arbitrary sequence one after the other by a 
user B of the speech processing system. For this case, a 

15 detection of the boundaries between the individual words is 

required. This can take place either by means of a method for 
the explicit detection of the word boundaries, or the word 
boundaries are defined by the word search algorithm. In the 
word search, all numerals in the input signal are examined in 

20 parallel, and various ends are hypothesized. At each end, a 
further numeral can be sought in the signal. For this 
purpose, a search space is constructed that pursues the 
various alternatives at the same time and orders them 
according to predeterminable criteria of quality. In this 

25 context, for example the principle of dynamic programming can 
be used, as described in document [1] . Such a criterion can 
be the difference of the observed parts of the signal from the 
reference patterns, the individually stored numerals. As a 
rule, numeral chain recognizers are distinguished from 

3 0 continuous recognizers by the particular type of modeling of 

the acoustic reference patterns, in which a specific manner of 
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articulation or typical lengths of the individual numerals are 
exploited. By means of a modeling of pauses in speech or by 
means of a decision criterion, e.g. the energy visible in the 
signal, the end of such a sequence can be detected in various 
5 heuristics. 

- Recognition of words from a limited vocabulary 

If a speech recognition module Ei is constructed as a 
speech recognizer for particular frequent command words from a 
limited vocabulary, called the core vocabulary, then 

10 algorithmically the corresponding method is applied as is also 
applied for individual numeral recognition, but with the 
difference that the vocabulary is expanded by the 
corresponding words, or, respectively, that the vocabulary 
differs from the numerals with the corresponding stored 

15 representatives of the individual words. The specially 
expanded or, respectively, replaced vocabulary is 
predetermined for the speech recognizer and as a rule is 
trained with a larger number of speakers. 

- Recognition of an individual word from an unlimited 
20 vocabulary 

For the recognition of speech spoken in a flowing manner 
with an unlimited vocabulary, a speech recognition module Ei 
is constructed in such a way that the words of the vocabulary 
are not known to the speech recognition algorithm that is 

25 realized in the speech recognition module Ei, but rather are 
determined beforehand by a user B. The algorithm for speech 
recognition is standardly based on the recognition of a 
sequence of phonetic units, called phonemes, or other word 
subunits from which the words of the vocabulary to be 

30 recognized can be composed. In the lexicon of the speech 

recognition module Ei, the phonetic sequences that identify 



9 



the respective word are stored in a memory. In their 
model ing, the sounds can also comprise further subunits, such 
as for example the beginning of the phoneme, the middle of the 
phoneme, the end of the phoneme, the transition of the 
5 phoneme, etc. The speech recognition method in turn searches 
the observed speech signal for the sequence with the smallest 
difference from the reference data. The individual word 
recognizer can algorithmically exploit the circumstance that a 
further sequence cannot follow any of the phonetic sequences 

10 stored in the stock, 

- Recognition of speech spoken in a flowing manner with an 
unlimited vocabulary 

If a speech recognition module Ei is constructed for the 
recognition of expressions of the user spoken in flowing 

15 fashion with an unlimited vocabulary, then as a rule the 

speech recognition module Ei differs from a speech recognition 
module Ei for numeral chain recognition in the type of 
modeling of the phonetic units. If the lexical inventory, 
i.e. the vocabulary, is unlimited, which is also designated an 

20 open lexical inventory, the acoustic modeling takes place 

mostly on phonetic units, the phonemes, or on word subunits 
from which the vocabulary to be recognized can be composed. 
As in the speech recognition module Ei for continuous 
numerals, in this construction of the speech recognition 

25 module the boundaries between the words are determined by a 
suitable search. This takes place for example by means of a 
modeling of speech pauses or by means of a decision criterion, 
e.g. the energy visible in the signal. The speech recognition 
module can output the most probable sequence of words, or, 

30 with expansions in the search algorithm, can output a 

predeterminable number of the most probable sequences of 
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words. In continuous speech recognition modules, the search 
is mostly supported by a modeling of the word sequences that 
are typical or possible for a particular speech segment, 
called language models. Language models are to be understood 
5 for example as grammatical modellings of the word sequences or 
word lists, or statistical modellings in the sense of what is 
called an N-gram analysis. 

- Recognition of predetermined word combinations 

For the recognition of predetermined word combinations, a 
10 speech recognition module El is constructed in such a way that 
for example date indications, time indications, measure 
indications, count indications, etc., are used as a special 
vocabulary. These speech recognition modules Ei are 
standardly speech recognition modules Ei for continuous 
15 speech, whose lexical inventory is however tailored to the 

predetermined word combinations from the respective regions, 
such as for example a date indication or a measure indication. 
In addition, the word sequence search can be supported by a 
modeling of the possible word sequences, as in a continuous 
20 speech recognition module Ei, whereby this modeling is 

distinguished in that it essentially takes into particular 
account the ways of expression that are to be expected in one 
of these verbal subregions, e.g. date indications. 

- Keyword recognition 

25 In a speech recognition module Ei for keyword 

recognition, one or several predetermined words previously 
stored in a lexicon can be detected in a speech input signal. 
The various possibilities for constructing the speech 
recognition module for keyword recognition are described for 

30 example in document [4] . 

- Recognition of keyword combinations 
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For the recognition of keyword combinations, the same 
principle is used as for the recognition of keywords, with the 
difference that pre-defined word chains can be detected in an 
arbitrary input signal ES . The word chains can be stored 
5 either explicitly as reference data or can be described by 
other methods, for example language models or grammars. 

- Alphabet recognition 

If a speech recognition module Ei is constructed as an 
alphabet recognition means, the letters are standardly 

10 described with particular acoustic phonetic models. In this 
case, the letters of the alphabet are essentially used as 
lexical units that are provided in a memory of the speech 
processing system. An alphabet recognizer can for example be 
realized as an individual word recognizer [..•] the isolated 

15 alphabet or also as a continuous recognizer for a flowing 

spelling. In continuous alphabet recognizers, the search for 
the actual word sequences can be supported by language models 
or also by grammars . 

- Phonetic sequence recognition 

20 A speech recognition module Ei for phonetic sequence 

recognition is distinguished in that the acoustic units are 
modeled on phonetic units, and the phonetic sequences to be 
recognized are not limited by a lexicon. Phonetic sequence 
recognizers or, respectively, phoneme recognizers are based on 

25 acoustic -phonetic modellings of the sounds of a language. The 
search in the acoustic speech signal is carried out with the 
phonetic inventory. An arbitrary chain of sounds is 
hypothesized. In the acoustic analysis, various heuristics 
are used in order to prevent a too -rapid exchange of the 

30 hypothesized sounds, the phonemes, in an expression. In 

addition, phonetic regularities of a language can be taken 
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into account as limitations of the sound sequences to be 
recognized, in the sense of language models. 

- DTMF recognition 

A DTMF recognition is known from the widest variety of 
5 telephonic speech recognition. 

As a speech output module Aj , various known principles 
can be used for the realization of various speech output 
principles : 

- Output of predetermined stored speech stocks 

10 If a speech output module Aj is constructed for the 

output of predetermined stored speech stocks, speech 
expressions previously spoken by an arbitrary person, recorded 
and stored in digitized form, are outputted via an arbitrary 
acoustic channel . The recorded expressions can hereby be the 

15 complete expressions to be outputted that are respectively 
provided for a particular situation, or else also fragments 
thereof that are put together during runtime to form complete 
expressions. A typical example of this is the formation of 
chains of numerals, whose length and composition cannot in 

20 most cases be foreseen. However, various intonational 

variants of one and the same expression may also be present 
according to context, and can be selected in a software 
program corresponding to the current context. In addition, 
various methods can be used that can carry out what is called 

25 an acoustic smoothing of the stocks used at the transitions of 
the concatenation of the individual speech stocks in the most 
various form. 

- Output of words synthesized from stored phonemes 

Possible realizations for the construction of speech 
30 output modules for the synthesis of stored phonemes to form 
synthesized words are described in document [5] . 
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The means MA for the selection of the speech recognition 
modules Ei or, respectively, speech output modules Aj 
respectively to be used in an application scenario is 
constructed in such a way that, dependent on an input signal 
5 ES supplied to the means MA, at least one of the speech 

recognition modules or, respectively, of the speech output 
modules is activated, and, insofar as is required, parameters 
required in the further method for speech recognition or, 
respectively, for speech output are supplied to the individual 

10 speech recognition modules Ei or, respectively, speech output 
modules Aj . The corresponding parameters can for example be 
that resources are assigned to the speech recognition modules 
Ei or, respectively, to the speech output modules Aj , or also 
for example that the memory addresses of the digital lexicons 

15 used are supplied to the speech recognition modules Ei or, 
respectively, to the speech output modules Aj * 

In this embodiment, the results, i.e. the outputted 
speech signals from the speech output modules Aj , are supplied 
to the means MA, and are outputted to the user B via for 

20 example an output AS of the speech processing system. A 

speech signal spoken by a user B is supplied to the means MA 
via an input E, for example realized by a microphone, and is 
supplied to the respectively activated speech recognition 
module Ei via the means MA, dependent on the respectively 

25 activated speech recognition modules Ei . 

The individual modules Ei, Aj use, among other things, 
knowledge sources, e.g. digital lexicons, general data stored 
in a memory of the speech processing system. The knowledge 
sources are for example also to be understood as including for 

30 example acoustic models specialized to the respective speech 
recognition task, e.g. an individual word recognizer with 
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sound models or individual word recognizer with acoustic 
models for arbitrary words. However, the knowledge sources 
can also be the specific vocabularies available for selection 
by the recognizers for a given dialog step in a dialog system, 
5 or for particular, expected input-specific speech models, as 
they are known. 

The vocabulary used by the speech recognition modules Ei 
is arbitrarily expandable by means of orthographic, i.e. 
speaker- independent , or verbal, i.e. speaker - dependent , input, 

10 both in production and also currently during the runtime of 

the speech processing system by means of an evaluation of the 
running input data, and is thus adaptable to changing demands. 

By means of suitable learning methods, the knowledge 
sources can be modified during the operation of the speech 

15 processing system, and can be adapted to the particular 
situation of use and application. 

In addition, a parameter that is supplied to the speech 
recognition modules Ei or to the speech output modules Aj can 
be that the parameter indicates whether speech signals spoken 

20 by the user B can be processed during a speech output itself 
and recorded by the speech processing system. 

In the following, a construction of the speech processing 
system as a speech dialog system is shown, on the basis of a 
very simple example for a dialog situation. 

25 In the following representation, in a pseudo-message flow 

diagram, at the beginning of a row on the left side of an 
arrow the sender of a particular signal is respectively 
described, and on the right side of an arrow the receiver of 
the signal that is further indicated in this row is 

3 0 respectively described. 
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In this example, a development that is described later, 
namely a dialog sequence control DA, is assumed, without 
limitation of generality. 

DA -4 MA: Output ("Please state your number after the tone") 
MA -4 Al: PlayPromptFile 

(Pleas e_state_your_numbe r _a f t e r_t he_ t one ) 
Al -4 MA: PlayPromptFinished 
MA -4 DA: Out put Complete 



10 DA -4 MA: RecognizeExpression (Domain=Numerals , Number max. =5) 
MA -4 El: BeginRecognition (Mode=lndividual numeral, 
Lexicon=Digits) 



MA -4 E2 : BeginRecognition (Mode=Continuous numeral 
15 recognition, Lexicon^Digits) 

El MA: RecognizeOutput (Mode=Individual numeral, Result=0) 
E2 MA: RecognizeOutput (Mode=continuous numerals, recognized 
numeral sequence: 12 3 4 5) 



MA 


-» 


DA: 


InputNumeral Sequence : (12 


3 4 


5) 


DA 


-4 


MA: 


Output ("Your number is 1 2 


3 4 


5") 


DA 


-4 


Al: 


PlayPromptFile (Your_number_ 


.is) 




MA 


-* 


A2 : 


Synthesize (Numeral Sequence 


1 2 


3 4 5) 


Al 


-4 


MA: 


PlayPromptFinished 






A2 


-4 


MA: 


Synthesize Finished 






MA 


-* 


DA: 


Output Compl e t e 







DA -4 MA: Output ("Do you wish to speak with sales or with 
customer service? " ) 
30 MA -4 Al: PlayPromptFile (Do_you_wish_to__speak_with__sales_or 

with customer service) 



16 



Al -4 MA: Prompt file not available 

MA -4 A2 : Synthesize (Do you wish to speak with sales or with 

customer service?) 
A2 -4 MA: Synthesis finished 
5 MA -4 DA: Output complete 

DA -4 DM; RecognizeSpeech (Domain=Sales_or_customer_service) 
MA -4 E3 : BeginRecognit ion (Mode=continuous , Lexicon=Core 
vocabulary) 

10 DM -4 E4 : BeginRecognit ion (Mode=keyword recognition, 

Lexicon=sales and customer service) 
E3 -4 MA: RecognizerOutput (Mode=continuous , Result^ 11 sales, 
please" ) 

E4 -4 MA: RecognizerOutput (Mode=keyword recognition, 
15 resul t = 11 sales " ) 

MA -4 DA: InputCharacterSequence ( sales) 

DA -4 MA: Output ( " Your date of birth, please") 

MA -4 Al: PlayPromptFile (Your_date_of_birth_please) 

20 

DA -4 MA: RecognizeExpression (Domain=Date) 
DM -4 E5 : BeginRecognit ion (Mode=continuous , Lexicon=date 
indication) 

E5 -4 MA: RecognizeOutput (Mode=Continuous, Result 1=" January 
25 first, nineteen-f if ty, " Result 2 = "lNrovember first, 

nineteen-f if ty" ) 
MA -4 DA: InputCharacterSequence (January first nineteen-f if ty) 
DA -4 MA: Output ("You were born on January first, nineteen- 
fifty") 

30 MA -4 A3 : ConcatOutput (You_were i _born_on January first, 

nineteen-f if ty) 
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A3 -4 MA: PlayPromptConcatComplete 

MA DA: Out put Complete 

DA MA: RecognizeSpeech (Domain=y/n question) 

MA E6 : BeginRecognition (mode=individual word, 

5 lexicon= (yes , no) ) 



In this simple example, the speech recognition modules Ei 
comprise the following simple speech recognition 
characteristics : 

10 The speech recognition module El is constructed as an 

individual numeral recognizer, E2 is constructed as a speech 
recognition module for the recognition of continuous numerals, 
i.e. chains of numerals. The speech recognition module E3 is 
constructed as a speech recognition module for the recognition 

15 of speech spoken in a flowing manner with a limited 

vocabulary, the lexicon "core vocabulary." The speech 
recognition module E4 is constructed as a keyword recognizer 
with the lexicon "sales and customer service, " and the speech 
recognition module E5 is constructed as a speech recognition 

20 module for recognizing predetermined word combinations; in 

this specific case it is constructed for the recognition of a 
date indication. 

In this simple example, the speech output modules Aj used 
are constructed as a module for the output of predetermined, 

25 stored speech stores (Al) , a module for synthesizing speech 
from for example phonemes (A2) , and a module for outputting 
compositions of individually predetermined stored speech 
stores (A3) . 

As can be seen from this simple example, the results of 
30 the speech recognition are supplied to the means MA by the 
speech recognition modules Ei, or, respectively, the speech 
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signal to be outputted is supplied to the means MA by the 
speech output modules Aj . 

On the other hand, the respective information required 
for the recognition is supplied to the individual modules Ei f 
5 Aj by the means MA, e.g. also the speech signal uttered into 
the speech processing system. 

A further development of the speech processing system and 
of the method for speech processing is that the speech signal 
is also simultaneously processed by several speech recognition 
10 modules Ei or speech output modules Aj , and that the 

respective best result with respect to an arbitrary similarity 
criterion is applied, or also only one result is applied in 
order to increase recognition security. 

Dialog sequence control 

15 With the dialog sequence control DA, a predeterminable 

dialog is carried out with the user B of the speech processing 
system according to the situation of application, and, 
dependent on the respective dialog situation, control 
information, e.g. information still required, is supplied to 

20 the means MA. In addition, in this development for example 
the result of the speech recognition modules is evaluated in 
the means for dialog sequence control, which can for example 
take place in the following manner: 

The dialog sequence control DA comprises at least the 

25 following components that are coupled with one another, e.g. 
via a bus BU (cf . Figure 2) : 

- an input means EM, 

- an input memory ESP, 

- an action determination unit AE, 
3 0 - a parameter parser PP. 
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The manner of operation of the individual components of 
the dialog sequence control DA will be explained in more 
detail below in connection with the methods for sequence 
control of a dialog with the user B. 
5 The action determination unit AE serves on the one hand 

for the determination of action indications and on the other 
hand for the determination of at least one predeterminable 
action that is indicated by the action indication. The action 
is determined from an arbitrary number of predeterminable 

10 actions- Various ways of proceeding for the determination of 
the action sought, which action is indicated by the action 
indication, are known to those skilled in the art. However, 
some possibilities, presented as examples, for the realization 
thereof will be explained in detail in connection with the 

15 method. 

The parameter parser PP serves for the determination of 

action parameters that are respectively allocated 

unambiguously to each action, from the digital action 

information stored in the input memory ESP. 
20 In Figure 2, the dialog sequence control DA is 

represented with some developments of the dialog sequence 

control DA for action determination. 

An information database ID provided in a construction 

respectively contains sought application-specific information 
2 5 that is determined on the basis of the determined action and 

the action parameters subsequently determined for the action. 
In a development, an action clarification unit AKE is 

provided for the determination of further action indications 

by the user. Above all, the action clarification unit AKE is 
30 of increased importance in areas of application where it is 

important to determine the sought action among various 
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actions. If the action indications handed over to the dialog 
sequence control DA by the user B are not sufficient for the 
determination of the action / the action clarification unit AKE 
is applied for the determination of further action indications 
5 by the user B. The construction of the action clarification 
unit AKE is explained in more detail in connection with the 
method. However, it is not necessary that the action 
clarification unit AKE determine exactly one action. It is 
likewise provided to reduce the number of predetermined 

10 actions in an arbitrary manner, in order in this way to 
further process only a smaller number of actions. 

In a development , a parameter clarification unit PKE is 
provided for the determination of further action parameters by 
the user B. The parameter clarification unit PKE is applied 

15 if for the at least one determined action not all action 
parameters allocated to the respective actions can be 
determined. For this case it is advantageous that the 
parameter clarification unit PKE is applied in order to 
determine missing action parameters from the user B. 

2 0 In a development of the dialog sequence control DA, it is 

additionally advantageous to provide a first action memory AS1 
and/or a second action memory AS 2 . In the first action memory 
AS1, actions and their action parameters are stored in which 
the at least one action indication agrees completely with at 

25 least one part of key terms whose meaning is described below. 
In the second action memory AS2, actions and the action 
parameters thereof are stored in which at least one of the 
action indications agrees with at least one key term. 

In addition, in a development a means GF is provided for 

30 the generation of questions (prompts) to the user B for the 
determination of further action indications and/or further 
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action parameters. The design of the means GF for generating 
questions is known for example from document [3] . 

In addition, in a development of the dialog sequence 
control DA it is provided to use a control unit STE for 
5 controlling a background application HA using the determined 
information. The construction of the control unit STE is 
completely application-specific, and serves only to apply the 
determined action for the controlling of the background 
application HA. The various possibilities of use, and thus 
10 also the various constructions of the control unit STE, are 
described in detail below. 

In addition, in a development a key term editor SE is 
provided with which on the one hand the key terms designed 
below, and on the other hand new actions and/or action 
15 parameters, can be modified, removed, or added. 

The individual steps of the method are shown in Figure 3 . 
In a first step 3 01, action information comprising at 
least one action indication is handed over by the user B of 
the dialog sequence control DA. The method need not 
2 0 necessarily be carried out with the application described 

above; it is also possible to execute the method with the aid 
of a computer. 

The action information is handed over by the user B to 
the dialog sequence control DA or to the speech processing 
25 system. The action information is digitized and stored as 
digitized action information 302. The digital action 
information can for example be stored in a phoneme 
representation or also in the form of a standard code for the 
representation of written characters (e.g. ASCII code). 
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In a further step 303, at least one action indication is 
determined from the stored action information using the 
parameter parser PP, whose design is known for example from 
document [4] . 

5 Using the action indication and the action determination 

unit AE, at least one action is determined 304. 

The determination of the at least one action can take 
place in various ways. One very simple , but in no way 
exclusively usable, method is to allocate to each action a 

10 predeterminable number of key terms that respectively 

characterize the action, and to compare the action indications 
determined from the action information, determined by the 
parameter parser PP, with the key terms. The comparison can 
take place by means of a direct word comparison or also by 

15 means of an arbitrary pattern comparison whose algorithms are 
familiar to anyone skilled in the art. 

If the at least one action has been determined 3 04, then 
for all actions the action parameters that are unambiguously 
allocated to the actions are determined from the digital 

20 action information using the parameter parser PP 305. On the 
basis of the determined action, as well as of the associated 
action parameters, the action is now carried out in a final 
step 306, 

In a development of the method, information concerning 
25 the action is determined from the information database ID, and 
is represented to the user B via the output unit AUS. 

If the action indications contained in the digital action 
information are not sufficient for the determination of at 
least one action 401, it is provided in a development to 
3 0 determine further action indications for example by means of a 
dialog with the user B 402. 
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This takes place for example using the action 
clarification units AKE, which is [sic] preferably constructed 
in such a way that for the predetermined actions at least one 
determination prompt is respectively unambiguously allocated. 
5 The determination prompt is to be understood for example as 

questions that are allocated to the respective action and are 
fixedly stored. These determination prompts are represented 
to the user B in the case of insufficient information for the 
determination of the actions. The user B gives responses to 

10 these determination prompts, i.e. to the corresponding 

questions, which responses are then in turn processed, whereby 
further action indications are determined. The dialog is 
carried out and repeated until the determined action 
indications and further action indications are sufficient for 

15 the identification of the predeterminable number of actions. 

If the at least one action is determined, then for each 
action the action parameters allocated to the respective 
action are determined 3 05. 

If, however, not all required action parameters are 

20 completely known or have been completely determined from the 

digital action information determined in the input memory ESP, 
which is checked in an additional method step 403, then the 
remaining action parameters are for example determined by the 
parameter clarification unit PKE 404. 

25 This also takes place for example in a dialog with the 

user B. For this dialog, prompts allocated to the individual 
parameters, called parameter prompts in the following, can in 
turn be used, which, given the necessary determination of the 
corresponding action parameters, can be outputted to the user 

30 B, and the user B responds to the corresponding questions. 
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Both the determination of further action indications 402 
and also the determination of the remaining action parameters 
4 04 can however also take place in a different way. 

For example, it is not absolutely necessary to allocate 
5 fixedly predetermined determination prompts or parameter 
prompts to the individual actions or, respectively, 
parameters, whereby on the one hand the simplicity in the 
execution of the method and in the adaptation of the 
respective application and the expandability of the actions 
10 and parameters is indeed considerably simplified, but the user 
B is always given only fixedly predetermined questions. 

In a development of the method and of the dialog sequence 
control DA, it is provided to use the means GF for the 
generation of questions to the user B. A possibility for the 
15 realization of the means GF is for example described in 
document [3] . 

In a development of the method, it is provided to execute 
the following steps for the determination of the action in the 
action determination unit AE. In the action determination 
20 unit AE, for each action the comparison of the key terms with 
the action indications is carried out. 

A first set of actions is determined in which all action 
indications agree with at least a part of the key terms . 
Graphically described, this procedure means that the 
25 intersection of all actions indicated by the action 

indications is determined. The first set of actions is for 
example stored in the first action memory AS1. 

In addition, a second set of actions is formed in which 
at least one part of the action indications agrees with the 
30 key terms. This procedure means, graphically, a formation of 
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the union of all actions indicated. The second set of actions 
is for example stored in the second action memory AS2 . 

If the first set contains exactly one element, the action 
to be determined is identified unambiguously. For this case, 
5 no further clarification dialog is required. If, however, the 
first set contains more elements than the number of actions to 
be determined, the action is not specified precisely enough by 
the initial user input. In this case, it is advantageous to 
execute the clarification dialog described above. 

10 If the first set is an empty set, the initial user 

expression contains indications of disjoint actions. In this 
case, the clarification dialog is advantageously controlled by 
the actions contained in the second set. 

If both the first set and also the second set are empty 

15 sets, then for example a clarification dialog is carried out 
for distinguishing all actions known in the application. 

Thus, the first set and the second set in this 
development serve as a foundation for the dialog with the user 
B for the determination of the at least one action. 

20 In a development of the method, the determined action can 

advantageously be used for the controlling of at least one 
background application HA. 

The background application HA is to be understood as the 
widest variety of possibilities of use of the method and of 

25 the dialog sequence control DA, which are explained below. 

The method and the dialog sequence control DA can for 
example advantageously be used in the following areas: 

- in a telephone service, 

3 0 - in a t el epur chasing system and/or telebanking system, 

- in what is known as a voice mail handling system, 
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- for controlling an arbitrary machine, e.g. a machine tool 
or a robot , 

- for controlling a computer, 

- in a messaging system of a branch exchange of a 
5 telecommunication system. 

In addition, in a development of the method it is 
advantageous that the evaluation of the digital action 
information that is stored in the input memory ESP is 

10 determined by the parameter parser PP in a sequence that is 
inverted in relation to the input of the action information. 

This means that the evaluation of the action information 
at the conclusion of the action information is begun and is 
continued until the beginning of the action information. By 

15 means of this procedure, a correction of the action 

information by the user B is possible in a sentence, which is 
explained in more detail below. 

For easier understanding of the dialog sequence control 
DA, in the following a very simple example from the area of 

20 tourism is presented. For example, in this area of 
application the following actions are defined: 

- Information about train connections; 

- Information about flight connections; 
25 - Information about boat connections; 

- Information about hotel rooms; 

- Restaurant information. 

The individual actions can for example be characterized 
3 0 by the following parameters: 
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Train information: <departure locations <arrival 
location>, <date>, <time> 

Flight information: <departure location>, <arrival 
location>, <date>, <tirae> 

Boat information: <departure location>, <arrival 
location> / <date>, <time> 

Hotel information: <location>, <room>, <type>, <number of 

persons>, < check- in date>, <check-out date> 

Restaurant information: <location> / <style>, <category>. 



In the following, the individual actions are specified in 
the form of a pseudo-code. A number is hereby respectively 
unambiguously assigned under point 2.1 to the action, under 
point 2.2 to the respective determination prompt and under 
15 number 2.3 to the individual parameter specifications, which, 
dependent respectively on the defined grammar in this case for 
predetermined parameter prompts, are represented in such a way 
that the respective grammar of the respective action parameter 
is respectively given under point 3.1 and point 3.3, and under 
20 point 3.2 the respective parameter prompt is shown, 
action (train information) 
{2.1: number = 1 

2.2: determination prompt = 'Would you like to request 
train information? 1 
25 2.3: Parameter specifications = {parameter 

{3.1: g_departure (X) 

3.2: 'Please state the 

desired departure location. ' 

3.3: g_location (X) 
30 } 

parameter 
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the desired arrival location. 1 



you wish to travel?' 



15 you wish to travel? 



{3.1: g_inf ormation (Y) 
3.2: 'Please state 

3.3: g_location (Y) 

} 

parameter 
{3.1: g_date(Z) 

3.2: 'On which date do 

3.3: g_date(Z) 
} 

parameter 
{3.1: g_time(W) 

3.2: 'At what time do 

3.3: g_time(W) 
} 



} 



2,4: Action indications = {from, to, travel, train 
20 information, railway, railway information} 
2.5: Confirmation prompt = 

'You are receiving information about trains 
from X to Y on Z at W o'clock. 1 

} 

25 

action (flight information) 
{2.1: Number = 2 
2.2: Determination prompt = 'Would you like to request 
flight information? ' 
30 2.3: Parameter specifications = {parameter 

{3.1: g_departure (X) 

29 



10 



15 



25 



desired departure location. 



the desired arrival location. ' 



you wish to fly? ' 



20 you wish to fly? 1 



} 



3.2: 'Please state the 
3.3: g_location (X) 

} 

parameter 

{3.1: g_inf ormation (Y) 

3.2: 'Please state 

3.3: g_location (Y) 

} 

parameter 
{3.1: g_date(Z) 

3.2: r On which date do 

3.3: g_date(Z) 
} 

parameter 
{3.1: g_time(W) 

3.2: ' At what time do 

3.3: g_time(W) 
} 



2.4: Action indications = {from, to, travel, fly, flight 
inf ormat ion , aircraf t } 
2.5: Confirmation prompt = 

'You are receiving information about 
flights from X to Y on Z at W o'clock. 1 

} 



30 



30 



10 



15 



25 



30 



Action (Boat information) 
{2.1: Number = 3 
2.2: Determination prompt = 'Would you like to receive 
boat information? ' 

2.3: Parameter specifications = {parameter 

{3.1: g_departure (X) 



desired departure location. f 



the desired arrival location. 1 



20 you wish to travel? 1 



you wish to travel? 1 



3.2: 1 Please state the 
3.3: g_location (X) 

} 

parameter 

{3.1: g_inf ormation ( Y) 

3.2: 1 Please state 

3.3: g_location (Y) 

} 

parameter 
{3.1: g__date(Z) 

3.2: 'On which date do 

3.3: g__date(Z) 
} 

parameter 
{3.1: g_time(W) 

3.2: 'At what time do 

3.3: g__time(W) 
} 



} 

2.4: Action indication = {from, to, travel/ boat information, 
boat} 
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2.5: Confirmation prompt = 

'You are receiving information about the boat 
connections from X to Y on Z at W o'clock. 1 
} 

5 

Action (Hotel information) 
{2.1: Number = 4 
2.2: Determination prompt = 'Would you like to request 
hotel information? ' 
10 2.3: Parameter specifications = {parameter 

{3.1: g_location indication (X) 

3.2: 1 In what location are you 

looking for a hotel? ' 

3.3: g_location indication (X) 

parameter 

{g__type of room(Y) 
3.2: 'Are you looking for a 

single room or a double room? 1 
20 3.3: g__type of room(Y) 

} 

parameter 

{3.1: g__number of persons (Z) 

3.2: 1 For how many persons are 

25 you looking for a room? r 

3.3: g__number of persons (Z) 

} 

parameter 

{3.1: g__f rom_date (W) 
30 3.2: 'When do you wish to check 

in? ' 
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3.3: g_date (W) 
} 

parameter 
{3.1: g_to_date(V) 
5 3.2: 'When do you wish to check 

out? ' 

3.3: g_date (V) 
} 

} 

10 2.4: Type of action = {Hotel, accommodation, single room, 

double room, room} 

2.5: Confirmation prompt - 

'You are receiving information about hotels in X that 

have Y available for Z persons from W to V. 1 
15 } 

Action (Restaurant information) 
{2.1: number = 5 
2.2: Determination prompt = 'Would you like to request 
20 restaurant information? 1 

2.3: Parameter specifications = {parameter 

{3.1: g_location indication (X) 
3.2: 'In which 

location do you wish to dine? 1 
25 3.3: g__location (X) 

} 

parameter 
{3.1: g_style{Y) 
3.2: 'Please state the desired 

30 style. ' 

3.3: g_style(Y) 

33 



} 

parameter 

{3.1: g_category (Z) 

3.2: 'Please state the desired 

5 category. ' 

3.3: g_category (Z) 

} 

} 

2.4: Action indication = {Restaurant, dine} 
10 2.5: Confirmation prompt = 

'You are receiving information about Y restaurants 
of category Z in X. 1 

} 

15 The following table shows examples of key terms for the 

individual actions that are unambiguously identified by the 
corresponding number in the curved brackets . 



from 


{i, 


2, 


3} 


to 


{i, 


2, 


3} 


travel 


{i, 


2, 


3} 


train information 


{i} 






railway 


{i} 






fly 


{2} 






flight information 


{2} 






aircraft 


{2} 






boat information 


{3} 






boat 


{3} 






hotel 


{4} 






accommodat ions 


{5} 






single room 


{4} 
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double room 

restaurant 

dine 



{4} 
{5} 
{5} 



5 An example of a dialog of the user B with the dialog 

sequence control DA is presented in the following. 

The user B expresses his desire in a precise and complete 
manner . 

The user B inputs the following information: 
10 "I would like to travel by train from Munich to Hamburg on 
January 1 st , 1996 at 5:00." 

The first action memory AS1 contains the first set of 
actions {l} using the above-described key terms and grammars. 
The second action memory AS2 contains the second set of 
15 actions {l, 2, 3}. 

Since the first action memory AS1 comprises exactly one 
element , the sought information is determined directly by the 
dialog sequence control DA from the information database ID, 
and is for example represented to the user B by means of the 
20 following response: 

"You are receiving information about the flights [sic] from 
Munich to Hamburg at 5:00 on January 1 st , 1996: 1 st flight, 2 nd 
flight, 3 rd flight, etc." 

In a further example, the user B states neither the time 
25 nor the means of transportation: 

The action information is for example the following: 
"I would like to travel from Munich to Hamburg on January 1 st , 
1996. » 

In this case, the first action memory AS1 contains the 
30 first set of actions {l, 2, 3}. The second action memory AS 2 
contains the second set of actions {l, 2, 3}. In the dialog 
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sequence control DA, a clarification dialog is initiated in 
order to distinguish between the actions {l, 2, 3}. 

For this purpose, the above-described determination 
prompt : 

5 "Would you like railway information? 11 
is outputted to the user B. 

The user B responds for example with "No. " 

In this case, the following determination prompt is for 
example outputted by the dialog sequence control DA: "Would 
10 you like flight information?" 

If the user B answers "yes" to this question, a further 
clarification dialog is carried out in order to determine the 
remaining still -missing action parameters for the flight 
information. The parameter prompt: 
15 "At what time do you wish to fly?" 

is hereby for example outputted to the user B for the 
determination of the action parameter time. 

The user B responds e.g. : 
"At five o'clock." 

2 0 The dialog sequence control DA now has sufficient 

information to determine the sought action. The result is for 
example represented to the user B in the following manner: 
"You are receiving information about the flights from Munich 
to Hamburg on January 1 st , 1996 at five o'clock." 
25 Further details concerning a possible construction of the 

dialog sequence control DA are described in the German patent 
application with official file number 19615693.9. 

Fig. 4 shows, in a flow diagram, the method in its method 
steps. The input signal ES is analyzed, whereby the input 

3 0 signal ES describes what type of speech recognition and/or 
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what type of speech output is required in the rest of the 
method 4 01. 

In addition, dependent on the input signal ES at least 
one speech recognition module Ei and/or one speech output 
5 module Aj are activated 402. 

In a final step 403, the respective speech recognition 
or, respectively, speech output is carried out in at least one 
of the speech recognition modules and/or speech output 
modules . 

10 In addition, in a development it is provided to carry 

out, in a separate pre-processing module W, all pre- 
processing steps or also all parts of algorithms that are 
simultaneously used in common in different methods for speech 
recognition. 

15 In the common pre-processing, for example in at least one 

pre-processing module W, it is for example possible to 
realize an acoustic pre-processing or what is known as a 
difference calculation of the examined parts of the speech 
signal . 

20 In addition, in a development it is advantageous to use 

resources in common, for example in such a way that a lexicon 
of several speech recognition modules Ei is used in common. 
This is for example possible in that a lexicon for the 
recognition of speech spoken in a flowing manner contains 

25 among other things the words "yes" and "no," whereby it is 
possible that in this case a keyword recognizer (keyword 
spotter) can also access the terms "yes" and "no" in the same 
lexicon for the execution of the speech recognition. 

A common use of the resources in the post -processing is 

30 also provided as a construction both in the method and also 

with a post-processing module in the speech processing system. 
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Both the speech processing system and the method for 
speech processing can be applied to great advantage both in 
speech recognition and in speech output as well as in the 
carrying out of a dialog with a user B, and also for the 
production of speech dialog systems, since a very simple 
combination of the application-specific combinations of speech 
recognition modules and/or speech output modules is possible. 
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1 . Speech processing system 

- having several speech recognition modules (Ei, i=l..n) and/or 
several speech output modules (Aj , j=l..m) that are respectively 
constructed specifically for a particular type of speech 
recognition and/or speech output, and 

- having a means (MA) for selecting at least one speech 
recognition module (Ei) and/or speech output module (Aj ) for a 
speech recognition and/or speech output to be carried out later, 
dependent on an input signal (ES) with which it is described what 
type of speech recognition and/or speech output is further 
required. 



15 2. Speech processing system according to claim 1, 
% in which the means (MA) for selecting the speech recognition 

3 module (Ei) and/or the speech output module (Aj ) is constructed 

[J in such a way that the speech recognition module (Ei) and/or the 

% speech output module (Aj ) is controlled by the means (MA) . 

*E 20 

3. Speech processing system according to claim 1 or 2 , 

in which parts of the speech recognition module (Ei) and/or of 
the speech output module (Aj ) that are used in common are 
realized in at least one pre-processing module (W) and/or in at 
25 least one post -processing module. 

4. Speech processing system according to claim 3, 

in which several speech recognition modules (Ei) and/or speech 
output modules (Aj ) use common resources. 
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5. Speech processing system according to one of claims 1 to 
4, in which a dialog sequence control (DA) is provided with 
which a dialog of the speech processing system with a user (B) 
of the speech processing system is realized. 

5 

6 . Speech processing system according to one of claims 1 to 
5, 

in which the input signal (ES) is produced by the dialog 
sequence control (DA) . 

10 

7. Speech processing system according to one of claims 1 to 
6, 

- in which the speech recognition module (Ei) is constructed 
at least for the execution of one of the following types of 
15 speech recognition: 

Individual numeral recognition, 
-- Recognition of chains of numerals, 
-- Recognition of words from a limited vocabulary, 
-- Recognition of an individual word with an unlimited 
20 vocabulary, 

Recognition of speech spoken in a flowing manner with an 
unlimited vocabulary, 

Recognition of predetermined word combinations, 

Keyword recognition, 
25 -- Alphabet recognition, 

-- Sound sequence recognition, 

Speech recognition, 
-- DTMF recognition, 
and/ or 
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1 ^f y r> X 

Replacement Page 

- in which the speech output module (Aj ) is constructed at least 
for the execution of one of the following types of speech output: 

Output of predetermined stored speech stores, 
5 Output of combined individual predetermined stored speech 

stores , 

- Output of words synthesized from stored phonemes, 
Output of DTMF tones. 

10 8. Method for speech processing, 

- in which an input signal (ES) is analyzed (401) , with which 
C3 signal it is described what type of speech recognition and/or 
nj speech output is further required, 

- in which, dependent on the input signal (ES) , from a set of 
^15 several speech recognition modules (Ei, i = l..n) and/or speech 
fU output modules (Aj , j = l..m), specifically constructed 

f*i respectively for a particular type of speech recognition and/or 

JS 1 speech output, at least one speech recognition module (Ei) and/or 

fU at least one speech output module (Aj ) is activated (402), and 

; J 20 - in which the respective type of speech recognition and/ or 

speech output is executed with the selected speech recognition 

module and/or speech output module (4 03) . 

9 . Method according to claim 8 , 

25 in which a pre-processing (W) of the speech signal to be 
recognized takes place before the speech recognition. 

10. Method according to claim 8 or 9, 

in which after the speech recognition a post-processing of the 
30 recognized speech signal takes place. 
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Abstract 



Speech processing system and method 

5 A speech processing system comprises an arbitrary number 

of speech recognition modules (Ei, i = l..n) and speech output 
modules (Aj , j - l..m). The modules provided respectively for 
a particular type of speech recognition or, respectively, 
speech output are selected and activated and parameterized 
10 according to the situation of application, dependent on an 

input signal (ES) , by a means (MA) for selecting the modules, 
so that the selected modules are applied for speech 
recognition or, respectively, speech output corresponding to 
their construction. 

15 

Sig. Fig, 1 
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Declaration and Power of Attorney For Patent Application 
Erklarung Fur Patentanmeldungen Mit Vollmacht 

German Language Declaration 



Als nachstehend benannter Erfinder erklare ich 
hiermit an Eides Statt: 



dass mein Wohnsitz, meine Postanschrift, und meine 
Staatsangehorigkeit den im Nachstehenden nach 
meinem Namen aufgefuhrten Angaben entsprechen, 



dass ich, nach bestem Wissen der ursprungliche, 
erste und alleinige Erfinder (falls nachstehend nur ein 
Name angegeben ist) oder ein ursprunglicher, erster 
und Miterfinder (falls nachstehend mehrere Namen 
aufgefuhrt sind) des Gegenstandes bin, fur den dieser 
Antrag gestellt wird und fur den ein Patent beantragt 
wird fur die Erfindung mit dem Titel: 

Sprachverarbeitungsverfahren und Ver- 



ahren zur Sprachverarbeitung 



deren Beschreibung 

zutreffendes ankreuzen) 
\E\ hier beigefugt ist. 

□ 



am 



als 



PCT internationale Anmeldung 

PCT Anmeldungsnummer 

eingereicht wurde und am . 



abgeandert wurde (falls tatsachlich abgeandert). 



Ich bestatige hiermit, dass ich den Inhalt der obigen 
Patentanmeldung einschliesslich der Anspruche 
durchgesehen und verstanden habe, die eventuell 
durch einen Zusatzantrag wie oben erwahnt abgean- 
dert wurde. 



Ich erkenne meine Pfiicht zur Offenbarung irgendwel- 
cher Informationen, die fur die Prufung der vorliegen- 
den Anmeldung in Einklang mit Absatz 37, Bundes- 
gesetzbuch, Paragraph 1.56(a) von Wichtigkeit sind, 
an. 



Ich beanspruche hiermit auslandische Prioritatsvor- 
teile gemass Abschnitt 35 der Zivilprozessordnung 
der Vereinigten Staaten, Paragraph 119 aller unten 
angegebenen Auslandsanmeldungen fur ein Patent 
oder eine Erfindersurkunde, und habe auch alle 
Auslandsanmeldungen fur ein Patent oder eine 
Erfindersurkunde nachstehend gekennzeichnet, die 
ein Anmeldedatum haben, das vor dem Anmelde- 
datum der Anmeldung liegt, fur die Prioritat bean- 
sprucht wird. 



As a below named inventor, I hereby declare that: 



My residence, post office address and citizenship are 
as stated below next to my name, 



I believe I am the original, first and sole inventor (if 
only one name is listed below) or an original, first and 
joint inventor (if plural names are listed below) of the 
subject matter which is claimed and for which a 
patent is sought on the invention entitled 



the specification of which 

Beck one) 
is attached hereto. 

was filed on _ 

PCT international application 

PCT Application No. 

and was amended on 



as 



(if applicable) 



I hereby state that I have reviewed and understand the 
contents of the above identified specification, 
including the claims as amended by any amendment 
referred to above. 



I acknowledge the duty to disclose information which 
is material to the examination of this application in 
accordance with Title 37, Code of Federal 
Regulations, §1 .56(a). 



I hereby claim foreign priority benefits under Title 35, 
United States Code, §119 of any foreign 
application(s) for patent or inventor's certificate listed 
below and have also identified below any foreign 
application for patent or inventor's certificate having a 
filing date before that of the application on which 
priority is claimed: 
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Patent and Trademark Office-U.S. DEPARTMENT OF COMMERCE 



Prior foreign appplications 
Prioritat beansprucht 



196 35 754.3 Germany 



(Number) 
(Nummer) 



German Language Declaration 



03. September 1996 



Priority Claimed 



m □ 



(Country) 
(Land) 



(Day Month Year Filed) 
(Tag Monat Jahr eingereicht) 



Yes 
Ja 



No 
Nein 



□ □ 



(Number) 
(Nummer) 



(Country) 
(Land) 



(Number) 
(Nummer) 



(Country) 
(Land) 



(Day Month Year Filed) 
(Tag Monat Jahr eingereicht) 



(Day Month Year Filed) 
(Tag Monat Jahr eingereicht) 



Yes 
Ja 



No 
Nein 



□ □ 



Yes 
Ja 



No 
Nein 



Ich beanspruche hiermit gemass Absatz 35 der Zivil- 
prozessordnung der Vereinigten Staaten, Paragraph 
120, den Vorzug aller unten aufgefuhrten Anmel- 
dungen und falls der Gegenstand aus jedem 
Anspruch dieser Anmeldung nicht in einer fruheren 
amerikanischen Patentanmeldung laut dem ersten 
Paragraphen des Absatzes 35 der ZivilprozeGordnung 
der Vereinigten Staaten, Paragraph 122 offenbart ist, 
erkenne ich gemass Absatz 37, Bundesgesetzbuch, 
Paragraph 1 .56(a) meine Pflicht zur Offenbarung von 
Informationen an, die zwischen dem Anmeldedatum 
der fruheren Anmeldung und dem nationalen oder 
PCT internationalen Anmeldedatum dieser Anmel- 
dung bekannt geworden sind. 



I hereby claim the benefit under Title 35. United 
States Code. §120 of any United States application(s) 
listed below and, insofar as the subject matter of each 
of the claims of this application is not disclosed in the 
prior United States application in the manner provided 
by the first paragraph of Title 35, United States Code, 
§122, I acknowledge the duty to disclose material 
information as defined in Title 37, Code of Federal 
Regulations, §1 .56(a) which occured between the 
filing date of the prior application and the national or 
PCT international filing date of this application. 



(Application Serial No.) 
(Anmeldeseriennummer) 



(Application Serial No.) 
(Anmeideseriennummer) 



(Filing Date) 
(Anmeldedatum) 



(Filing Date) 
(Anmeldedatum) 



(Status) 

( paten tiert, anhangig, 
aufgegeben) 



(Status) 

(patented, pending, 
abandoned) 



(Status) 

(patentiert, anhangig, 
aufgeben) 



(Status) 

(patented, pending, 
abandoned) 



Ich erklare hiermit, dass alle von mir in der vorliegen- 
den Erkiarung gemachten Angaben nach meinem 
besten Wissen und Gewissen der vollen Wahrheit 
entsprechen, und dass ich diese eidesstattliche Erkia- 
rung in Kenntnis dessen abgebe, dass wissentlich und 
vorsatzlich falsche Angaben gemass Paragraph 1001, 
Absatz 18 der Zivilprozessordnung der Vereinigten 
Staaten von Amerika mit Geldstrafe belegt und/oder 
Gefangnis bestraft werden koennen, und dass 
derartig wissentlich und vorsatzlich falsche Angaben 
die Giiltigkeit der voriiegenden Patentanmeldung oder 
eines darauf erteilten Patentes gefahrden konnen. 



I hereby declare that all statements made herein of 
my own knowledge are true and that ail statements 
made on information and belief are believed to be 
true, and further that these statements were made 
with the knowledge that willful false statements and 
the like so made are punishable by fine or 
imprisonment, or both, under Section 1001 of Title 18 
of the United States Code and that such willful false 
statements may jeopardize the validity of the 
application or any patent issued thereon. 
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German Language Declaration 



VERTRETUNGSVOLLMACHT: Ais benannter Erfinder 
beauftrage ich hiermit den nachstehend benannten 
Patentanwalt (oder die nachstehend benannten 
Patentanwalte) und/oder Patent-Agenten mit der 
Verfolgung der vorliegenden Patentanmeldung sowie 
mit der Abwicklung aller damit verbundenen 
Geschafte vor dem Patent- und Warenzeichenamt: 
(Name und Registrationsnummer anfuhren) 



POWER OF ATTORNEY: As a named inventor, I 
hereby appoint the following attorney(s) and/or 
agent (s) to prosecute this application and transact all 
business in the Patent and Trademark Office 
connected therewith, (list name and registration 
number) 



And I hereby appoint 

Messrs. John D. Simpson (Registration No. 19,842) Lews T. Steadman (17,074), William C. Stueber ( 16,453) , P. Phillips Connor £19,259), Dennis A. Gross 
( 24,410), Marvin Moody £l&5dS), Steven H. Noll (28T&82 ), Brett A. ValiqueT ffiffill Thomas I. Ross (29£75)7Kevin W. Guynn (29,927), Edward A. Lehmann 
*(^5gj; James D. Hobart (24.149V Robert M."Barre1T(30 l 142) J James Van Santen (16,584), J. Artfiur Gross (13,615), Richard J. Schvwarz (1^4^) and 
TffleWTrfA. Robinson (31 ,870 }^ David R. Metzger (32,919), John R. Garrett (27,888 ) all members of the firm of Hill, Steadman & Simpson, A Professional 
Corporation. 



Telefongesprache bitte richten an: Direct Telephone Calls to: (name and teiephone 

(Name und Telefonnummer) number) 

312/876-0200 

Ext. 



Postanschrift: Send Correspondence to: 

HILL, STEADMAN & SIMPSON 
^AProj^s^ional Corporation 
85th FloorJSeacsJTower. Chicago. Illinois 60606 



Voller Name des einzigen oder ursprunglichen Erfinders: 

NIEDERMAlR^Gerhard 


Full name of sole or first inventor; 


Unter§£tujft des Erfinde(sr\ Datum 


Inventor's signature Date 


D-85256^Lerkirchen, Germany 


Residence 


Staatsangehorigkeit 

Bundesrepublik Deutschland 


Citizenship 


Postanschrift 

Ludwig-Thoma-Str. 9 


Post Office Addess 


D-85256 Vierkirchen 
Bundesrepublik Deutschland 




Voiler Name des zweiten Miterfinders (falls zutreffend): 

KARSTEN. Peter 


Full name of second joint inventor, if any: 


Unterschrift des Erfinders Datum 


Second Inventor's signature Date 


Wohnsitz 

D-82061 Neuried, Germany T>BX 


Residence 


Staatsangehorigkeit 

Bundesrepublik Deutschland 


Citizenship 


Postanschrift 

Andreas Sammer Str. 2 


Post Office Address 


D-82061 Neuried 
Bundesrepublik Deutschland 





{Bitte entsprechende tnformationen und Unterschriften im (Supply similar information and signature for third and 



Falie von dritten und weiteren Miterfindern angeben). subsequent joint inventors). 
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Voller Name des dritten Miterfinders: 

HABENSEL, Steffen 


Full name of third joint inventor: 


Unterschrift des Erfinders Datum 


Inventor's signature 


Date 


Wohnsitz 

D-80689 Munchen. Germany VF~Y 


Residence 


Staatsangehdrigkeit " * 

Bundesrepublik Deutschland 




Posta n sch rift 

Mitterfeldstr. 59b 


Post Office Address 


u-<5Uboy Muncnen 
Bundesrepublik Deutschland 




Voller Name des vierten Miterfinders (falls zutreffend): 


Full name of fourth joint inventor, if any: 


Unterschrift des Erf inders Datum 


Inventor's signature 


Date 


Wohnsitz 

D-80798jyiunchen, Germany 7>^X 


Residence 


Staatsan geh origkeit 

Bundesrepublik Deutschland 


Citizenship 


Dn^fartQ/^h rift 

Gorresstr. 1 1 


Post Office Address 


p\ GXY70&L Ryii'mr^han 

Bundesrepublik Deutschland 




Voller Name des funtten Mitemnders (tails zutreitend). 


Full name of fifth joint inventor, if any; 


Unterschrift des Erfinders Datum 


Inventor's signature 


Date 


Wohnsitz 


Residence 


Staatsangehdrigkeit 


Citizenship 


D^&fro ne/^h rift* 


Post Office Address 






Voller Name des sechsten Miterfinders (falls zutreffend): 


Full name of sixth joint inventor, if any: 


Unterschrift des Erfinders Datum 


Inventor's signature 


Date 


Wohnsitz 


Residence 


Staatsangehdrigkeit 


Citizenship 


Paq4vi n *r h r iff 
rvo Leal ictl* I II II I 


Post Office Address 







(Bffte entsprechende tnformationen und Unterschriften im (Supply similar information and signature for third and 



Falie von dritten und weiteren Miterfmdern angeben). subsequent joint inventors). 
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